Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multi lora cpu #1

Closed
wants to merge 20 commits into from
Closed

Multi lora cpu #1

wants to merge 20 commits into from

Conversation

Akshat-Tripathi
Copy link
Collaborator

Adding

@Akshat-Tripathi
Added non-triton SGMV and BGMV ops (not kernels yet)
4a5df02 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Removed extra print
8c45f11 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Minor changes
676991b 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Made some minor shape-based fixes to the kernels
3457a73 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Added multi lora execution to the CPU backend
898e4e4 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Added __init__.py files
c537cc9 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Changed lora op importing to work irrespective of backend
8a17bc5 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Removed comment
733cecc 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Added multimodal lora support
5b8d04a 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Ran formatter
7ad9748 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Changed test backend back to nccl
d639b1a 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Updated lora tests to work with cpu
56c07d7 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Fixed lora op dtypes
4059ee7 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Updated tests to work with the CPU backend
9473042 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Removed redundant .squeeze call in lora ops
e00b6bd 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Stopped skipping some e2e tests
22fc007 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Fixed case where the lora_b tensor has 4 dims not 3
1272784 
Signed-off-by: Akshat Tripathi <[email protected]>
@Akshat-Tripathi
Formatted code
fb5e02f 
Signed-off-by: Akshat Tripathi <[email protected]>
@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 9, 2024

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@Akshat-Tripathi
Fixed formatting issues
c3e9afe 
Signed-off-by: Akshat Tripathi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Akshat-Tripathi